Grouping Users Based on User Profiles

ABSTRACT

The disclosure includes a system and method for grouping users into one or more categories for delivering targeted advertisements. A media application receives a captured image from a user, identifies an indexed image matching the captured image, retrieves metadata describing the captured image responsive to identifying the indexed image, generates a user profile for the user based on the metadata describing the captured image and a history of at least one previously captured image associated with the user, identifies a product in the captured image based on the metadata describing the captured image, provides the user with information about the product, updates the user profile based on how the user interacts with the product, and categorizes the user into one or more groups based on the user profile.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The specification relates to grouping of users based on their imagecapturing patterns across different media. In particular, thespecification relates to a system and method for generating userprofiles that group users into one or more categories.

2. Description of the Background Art

Users have a variety of interests that manifest in different mediums.For example, some users are more interested in financial informationwhen reading a newspaper and more interested in cooking techniques whenwatching television. Current techniques for generating user profilesfail to incorporate user activities that involve multiple mediums. As aresult, current techniques generate one dimensional profiles, such as auser that is only interested in financial information or only interestedin cooking techniques.

SUMMARY OF THE INVENTION

The disclosure includes a system and method for grouping users into oneor more categories for delivering targeted advertisement. In oneembodiment, the system includes a media application. The mediaapplication is configured to receive a captured image from a user,identify an indexed image matching the captured image, retrieve metadatadescribing the captured image responsive to identifying the indexedimage, generate a user profile for the user based on the metadatadescribing the captured image and a history of at least one previouslycaptured image associated with the user, identify a product in thecaptured image based on the metadata describing the captured image,provide the user with information about the product, update the userprofile based on how the user interacts with the product, and categorizethe user into one or more groups based on the user profile. In oneembodiment, the captured image can be from a video medium. In anotherembodiment, the captured image can be from a print advertising medium.

Other aspects include corresponding methods, systems, apparatuses, andcomputer program products for these and other innovative aspects.

The specification advantageously describes technology for generatinguser profiles that reflect a more complete version of the user based onprint and video media.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention is illustrated by way of example, and not by way oflimitation in the figures of the accompanying drawings in which likereference numerals are used to refer to similar elements.

FIG. 1 is a high-level block diagram illustrating one embodiment of asystem for grouping users for delivering targeted advertisements.

FIG. 2 is a block diagram illustrating one embodiment of a mediaapplication.

FIG. 3 is a flow diagram illustrating one embodiment of a method forindexing images.

FIG. 4 is a flow diagram illustrating one embodiment of a method forcategorizing the users into one or more groups.

FIG. 5 is a graphical representation of one embodiment that displaysinformation associated with identified products on a captured image.

FIG. 6 is a flow diagram illustrating one embodiment of a method fordelivering the user with an advertisement.

FIG. 7 is a graphical representation of one embodiment for deliveringadvertisement to the user capturing an image.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

A system and method for grouping users into one or more categories fordelivering targeted advertisements is described. In the followingdescription, for purposes of explanation, numerous specific details areset forth in order to provide a thorough understanding of the invention.It will be apparent, however, to one skilled in the art that theembodiments can be practiced without these specific details. In otherinstances, structures and devices are shown in block diagram form inorder to avoid obscuring the invention. For example, the invention isdescribed in one embodiment below with reference to user devices such asa smart phone and particular software and hardware. However, thedescription applies to any type of computing device that can receivedata and commands, and any peripheral devices providing services.

Reference in the specification to “one embodiment” or “an embodiment”means that a particular feature, structure, or characteristic describedin connection with the embodiment is included in at least oneembodiment. The appearances of the phrase “in one embodiment” in variousplaces in the specification are not necessarily all referring to thesame embodiment.

Some portions of the detailed descriptions that follow are presented interms of algorithms and symbolic representations of operations on databits within a computer memory. These algorithmic descriptions andrepresentations are the means used by those skilled in the dataprocessing arts to most effectively convey the substance of their workto others skilled in the art. An algorithm is here, and generally,conceived to be a self-consistent sequence of steps leading to a desiredresult. The steps are those requiring physical manipulations of physicalquantities. Usually, though not necessarily, these quantities take theform of electrical or magnetic signals capable of being stored,transferred, combined, compared, and otherwise manipulated. It hasproven convenient at times, principally for reasons of common usage, torefer to these signals as bits, values, elements, symbols, characters,terms, numbers or the like.

It should be borne in mind, however, that all of these and similar termsare to be associated with the appropriate physical quantities and aremerely convenient labels applied to these quantities. Unlessspecifically stated otherwise as apparent from the following discussion,it is appreciated that throughout the description, discussions utilizingterms such as “processing” or “computing” or “calculating” or“determining” or “displaying” or the like, refer to the action andprocesses of a computer system, or similar electronic computing device,that manipulates and transforms data represented as physical(electronic) quantities within the computer system's registers andmemories into other data similarly represented as physical quantitieswithin the computer system memories or registers or other suchinformation storage, transmission or display devices.

The invention also relates to an apparatus for performing the operationsherein. This apparatus may be specially constructed for the requiredpurposes, or it may comprise a general-purpose computer selectivelyactivated or reconfigured by a computer program stored in the computer.Such a computer program may be stored in a computer readable storagemedium, such as, but is not limited to, any type of disk includingfloppy disks, optical disks, CD-ROMs, and magnetic disks, read-onlymemories (ROMs), random access memories (RAMs), EPROMs, EEPROMs,magnetic or optical cards, flash memories including USB keys withnon-volatile memory or any type of media suitable for storing electronicinstructions, each coupled to a computer system bus.

Some embodiments can take the form of an entirely hardware embodiment,an entirely software embodiment or an embodiment containing bothhardware and software elements. A preferred embodiment is implemented insoftware, which includes but is not limited to firmware, residentsoftware, microcode, etc.

Furthermore, some embodiments can take the form of a computer programproduct accessible from a computer-usable or computer-readable mediumproviding program code for use by or in connection with a computer orany instruction execution system. For the purposes of this invention, acomputer-usable or computer readable medium can be any apparatus thatcan contain, store, communicate, propagate, or transport the program foruse by or in connection with the instruction execution system,apparatus, or device.

A data processing system suitable for storing and/or executing programcode will include at least one processor coupled directly or indirectlyto memory elements through a system bus. The memory elements can includelocal memory employed during actual execution of the program code, bulkstorage, and cache memories which provide temporary storage of at leastsome program code in order to reduce the number of times code must beretrieved from bulk storage during execution. Input/output or I/Odevices (including but not limited to keyboards, displays, pointingdevices, etc.) can be coupled to the system either directly or throughintervening I/O controllers.

Network adapters may also be coupled to the system to enable the dataprocessing system to become coupled to other data processing systems orremote printers or storage devices through intervening private or publicnetworks. Modems, cable modem and Ethernet cards are just a few of thecurrently available types of network adapters.

Finally, the algorithms and displays presented herein are not inherentlyrelated to any particular computer or other apparatus. Variousgeneral-purpose systems may be used with programs in accordance with theteachings herein, or it may prove convenient to construct morespecialized apparatus to perform the required method steps. The requiredstructure for a variety of these systems will appear from thedescription below. In addition, the specification is not described withreference to any particular programming language. It will be appreciatedthat a variety of programming languages may be used to implement theteachings of the various embodiments as described herein.

System Overview

FIG. 1 illustrates a block diagram of a system 100 for categorizingusers into one or more groups and determining targeted advertisementsfor delivery. The illustrated system 100 includes user devices 115 a . .. 115 n that can be accessed by users and a server 101. In FIG. 1 andthe remaining figures, a letter after a reference number, e.g., “115 a,”represents a reference to the element having that particular referencenumber. A reference number in the text without a following letter, e.g.,“115,” represents a general reference to instances of the elementbearing that reference number. In the illustrated embodiment, theseentities of the system 100 are communicatively coupled via a network105.

The network 105 can be a conventional type, wired or wireless, and mayhave numerous different configurations including a star configuration,token ring configuration or other configurations. Furthermore, thenetwork 105 may include a local area network (LAN), a wide area network(WAN) (e.g., the Internet), and/or other interconnected data pathsacross which multiple devices may communicate. In some embodiments, thenetwork 105 may be a peer-to-peer network. The network 105 may also becoupled to or includes portions of a telecommunications network forsending data in a variety of different communication protocols. In someembodiments, the network 105 includes Bluetooth communication networksor a cellular communications network for sending and receiving dataincluding via short messaging service (SMS), multimedia messagingservice (MMS), hypertext transfer protocol (HTTP), direct dataconnection, WAP, email, etc. Although FIG. 1 illustrates one network 105coupled to the user devices 115 and the server 101, in practice one ormore networks 105 can be connected to these entities.

The media player 107 may be a computing device including a processor, amemory and communication capabilities. In one embodiment, the mediaplayer 107 receives and playbacks multimedia content including audio andvideo, and is coupled to the network 105 via signal line 104. The mediaplayer 107 receives input commands from the user to play, pause or stopthe playback of multimedia content. For example, the media player 107can be a television including a set top box capable of receiving andplaying back content, such as movies, sitcoms, news, etc. from contentsources (i.e., basic cable, premium cable, satellite television, etc.)over the network 105. In another embodiment, the media player 107 doesnot have a monitor for outputting content. Accordingly, in someembodiments the media player 107 is communicatively coupled to a monitorfor outputting the content for display to the user. For example, themedia player 107 can be a Blu-ray™ device for playback ofhigh-definition content, a game console for high-definition interactivegameplay or for playback of high-definition content through on-demandinternet streaming (i.e., Netflix™, Amazon™, etc.) over the network 105.In one embodiment, the user devices 115 a . . . 115 n capture images ofcontent displayed by the media player 107. For example, capturing animage of a climactic scene from an action movie, capturing anadvertisement for car insurance, etc.

In one embodiment, the print media 109 can be a paper documentreproducing text and images in ink for user consumption. For example,the print media 109 can be a news magazine, a fashion magazine, a sportsmagazine, a tabloid magazine, a newspaper, a journal, etc. In oneembodiment, the user devices 115 a . . . 115 n capture images of contentdisplayed on print media 109. For example, capturing an image of a 2014electric car advertisement in a daily newspaper, capturing an image of acelebrity wearing a fashion leather jacket, etc.

In one embodiment, the media application 103 a may be operable on a userdevice 115 a, which is connected to the network 105 via signal line 106.In one embodiment, the user device 115 a, 115 n may be a computingdevice that includes a memory, a processor and a camera, for example alaptop computer, a desktop computer, a tablet computer, a mobiletelephone, a smartphone, a personal digital assistant (PDA), a mobileemail device, a webcam, a digital set top box (STB), digital videorecorder (DVR), connected television or any other electronic devicecapable of accessing a network 105. The user device 115 includes adisplay for viewing information provided by the server 101. The userdevices 115 a, 115 n in FIG. 1 are used by way of example. While FIG. 1illustrates two user devices 115 a and 115 n, the disclosure applies toa system architecture having one or more user devices 115.

In one embodiment, the media application 103 b may be operable on theserver 101, which is coupled to the network 105 via signal line 108. Theserver 101 may be a computing device including a processor, a memory andnetwork communication capabilities. The server 101 sends and receivesdata to and from other entities of the system 100 via the network 105.In one embodiment, the server 101 can be an advertisement server thatreceives and stores advertisements for marketing a variety of productcategories. The server 101 delivers the advertisements to the targetusers who have expressed interest in one or more product categories. Forexample, the server 101 receives data including a captured image of awrist watch worn by an actor on a video channel from the user device 115and sends targeted advertisement associated with the captured image ofthe wrist watch to the user device 115. In another example, the server101 receives data including a captured image of a smartphoneadvertisement in a magazine from the user device 115 and providesadditional information related to the smartphone advertisement to theuser device 115. In one embodiment, the images received by the server101 can also include an image copied from a website or an email or animage from any other source. While FIG. 1 includes one server 101, thesystem 100 may include one or more servers 101. The server 101 alsoincludes a data storage 243, which is described below in more detailwith reference to FIG. 2.

In one embodiment, the media application 103 acts as a thin-clientapplication that may be stored in part on the user device 115 and inpart as components of the server 101. For example, the media application103 a on the user device 115 could include software for capturing theimage and, in some embodiments, identify a product in the capturedimage. The media application 103 b on the server 101 matches the productidentified to retrieve targeted advertisements.

The media application 103 is code and routines for grouping users intoone or more categories for delivering targeted advertisements. In oneembodiment, the media application 103 can be implemented using hardwareincluding a field-programmable gate array (FPGA) or anapplication-specific integrated circuit (ASIC). In another embodiment,the media application 103 can be implemented using a combination ofhardware and software. In yet another embodiment, the media application103 may be stored in a combination of the devices and servers, or in oneof the devices or servers.

In one embodiment, the media application 103 receives a captured imagefrom a user, identifies an indexed image matching the captured image,retrieves metadata describing the captured image responsive toidentifying the indexed image, generates a user profile for the userbased on the metadata describing the captured image and a history of atleast one previously captured image associated with the user, identifiesa product in the captured image based on the metadata describing thecaptured image, provides the user with information about the product,updates the user profile based on how the user interacts with theproduct, and categorizes the user into one or more groups based on theuser profile.

In one embodiment, the media application 103 receives a captured imageincluding a product from a user, identifies an indexed image matchingthe captured image including the product, retrieves metadata describingthe captured image including the product responsive to identifying theindexed image, retrieves a user profile stored for the user, identifiesan advertisement from a set of advertisements based on the user profileand the metadata describing the captured image including the product andprovides the user with the advertisement based on the user profile.

In one embodiment, the metadata describes the captured image includes atime and location associated with the captured image, a genre of thecaptured image, a type of advertising medium associated with thecaptured image, a name of advertising medium associated with thecaptured image, and a name of the product in the captured image; theuser profile for the user includes age, gender, location, timestamp ofthe at least one previously captured image, metadata of the at least onepreviously captured image, history of products previously interactedwith by the user, history of products previously purchased by the userand time spent by the user shopping for products; and the user profilefor the user is generated at a time of initial user registration. Themedia application 103 provides the user with information about theproduct by displaying hotspots on the captured image linking theidentified product to the product's website; and categorizing the userinto one or more groups based on the user profile further comprisescategorizing by the user's time of engagement with an advertisingmedium. The media application categorizes the user into one or moregroups based on the user profile by categorizing by the user's rate ofproduct adoption; categorizing the user into one or more groups based onthe user profile further comprises categorizing by the user's shoppingpattern and the user's interest in products; and categorizing the userinto one or more groups based on the user profile further comprisescategorizing by the user's preference of a medium. The media application103 is described below in more detail with reference to FIG. 2.

Media Application

Referring now to FIG. 2, an example of the media application 103 isshown in more detail. FIG. 2 is a block diagram of a computing device200 that includes a media application 103, a processor 235, a memory237, a communication unit 241 and data storage 243 according to someexamples. The components of the computing device 200 are communicativelycoupled by a bus 220. The bus 220 may represent one or more busesincluding an industry standard architecture (ISA) bus, a peripheralcomponent interconnect (PCI) bus, a universal serial bus (USB), or someother bus known in the art to provide similar functionality. In someembodiments, the computing device 200 can be one of a user device 115and a server 101.

The processor 235 includes an arithmetic logic unit, a microprocessor, ageneral purpose controller or some other processor array to performcomputations and to optionally provide electronic display signals to adisplay device. The processor 235 is coupled to the bus 220 forcommunication with the other components via signal line 240. Theprocessor 235 processes data signals and may include various computingarchitectures including a complex instruction set computer (CISC)architecture, a reduced instruction set computer (RISC) architecture, oran architecture implementing a combination of instruction sets. AlthoughFIG. 2 includes a single processor 235, multiple processors 235 may beincluded. The processing capability may be limited to supporting thedisplay of images and the capture and transmission of images. Theprocessing capability might be enough to perform more complex tasks,including various types of feature extraction and sampling. It will beobvious to one skilled in the art that other processors, operatingsystems, sensors, displays and physical configurations are possible.

The memory 237 stores instructions and/or data that can be executed bythe processor 235. The memory 237 is coupled to the bus 220 forcommunication with the other components via signal line 242. Theinstructions and/or data may include code for performing the techniquesdescribed herein. The memory 237 may be a dynamic random access memory(DRAM) device, a static random access memory (SRAM) device, flash memoryor some other memory device known in the art. In some embodiments, thememory 237 also includes a non-volatile memory or similar permanentstorage device and media including a hard disk drive, a floppy diskdrive, a CD-ROM device, a DVD-ROM device, a DVD-RAM device, a DVD-RWdevice, a flash memory device, or some other mass storage device forstoring information on a more permanent basis.

The communication unit 241 is hardware for receiving and transmittingdata by linking the processor 235 to the network 105 and otherprocessing systems. The communication unit 241 receives data such asrequests from the user device 115 and transmits the requests tocomponents of the media application 103. The communication unit 241 alsotransmits additional information including targeted advertisements tothe user device 115 for display, for example, in response to receiving acaptured image of a product advertisement in print or video. Thecommunication unit 241 is coupled to the bus 220 via signal line 244. Inone embodiment, the communication unit 241 includes a port for directphysical connection to the user device 115 or to another communicationchannel. For example, the communication unit 241 includes an RJ45 portor similar port for wired communication with the user device 115. Inanother embodiment, the communication unit 241 includes a wirelesstransceiver (not shown) for exchanging data with the user device 115 orany other communication channel using one or more wireless communicationmethods, such as IEEE 802.11, IEEE 802.16, Bluetooth® or anothersuitable wireless communication method.

In one embodiment, the communication unit 241 includes a cellularcommunications transceiver for sending and receiving data over acellular communications network such as via short messaging service(SMS), multimedia messaging service (MMS), hypertext transfer protocol(HTTP), direct data connection, WAP, e-mail or another suitable type ofelectronic communication. In another embodiment, the communication unit241 includes a wired port and a wireless transceiver. The communicationunit 241 also provides other conventional connections to the network 105for distribution of files and/or media objects using standard networkprotocols such as TCP/IP, HTTP, HTTPS and SMTP as will be understood tothose skilled in the art.

The data storage 243 is a non-transitory memory that stores data forproviding the functionality described herein. The data storage 243 maybe a dynamic random access memory (DRAM) device, a static random accessmemory (SRAM) device, flash memory or some other memory devices. In oneembodiment, the data storage 243 also includes a non-volatile memory orsimilar permanent storage device and media including a hard disk drive,a floppy disk drive, a CD-ROM device, a DVD-ROM device, a DVD-RAMdevice, a DVD-RW device, a flash memory device, or some other massstorage device for storing information on a more permanent basis.

In the illustrated embodiment, the data storage 243 is communicativelycoupled to the bus 220 via signal line 246. In one embodiment, the datastorage 243 stores the user profiles of the users capturing images. Inone embodiment, the data storage 243 stores the categories of userprofiles for delivering targeted advertisement for effective productadvertising. In one embodiment, the data storage 243 stores the metadatadescribing the captured images. In another embodiment, the data storage243 stores the data in Extensible Markup Language (XML) file format.

In one embodiment, the media application 103 includes a controller 201,an image processor 203, a matching engine 205, a user profile engine207, a grouping engine 209, an advertisement engine 211, and a userinterface engine 213. The components of the media application 103 arecommunicatively coupled via the bus 220.

The controller 201 can be software including routines for handlingcommunications between the media application 103 and other components ofthe computing device 200. In one embodiment, the controller 201 can be aset of instructions executable by the processor 235 to provide thefunctionality described below for handling communications between themedia application 103 and other components of the computing device 200.In another embodiment, the controller 201 can be stored in the memory237 of the computing device 200 and can be accessible and executable bythe processor 235. In either embodiment, the controller 201 can beadapted for cooperation and communication with the processor 235 andother components of the computing device 200 via signal line 222.

In one embodiment, the controller 201 sends and receives data, via thecommunication unit 241, to and from one or more of user devices 115 andthe server 101. For example, the controller 201 receives, via thecommunication unit 241 a captured image from a user device 115 operatedby a user and sends the image to the matching engine 205. In anotherexample, the controller 201 receives graphical data for providing a userinterface to a user from the user interface engine 213 and sends thegraphical data to a user device 115, causing the user device 115 topresent the user interface to the user.

In one embodiment, the controller 201 receives data from othercomponents of the media application 103 and stores the data in the datastorage 243. For example, the controller 201 receives data includingmetadata of captured images from the image processor 203 and stores thedata in the data storage 243. In another embodiment, the controller 201retrieves data from the data storage 243 and sends the data to othercomponents of the media application 103. For example, the controller 201retrieves data including user profiles from the data storage 243 andsends the retrieved data to the grouping engine 209.

The image processor 203 can be software including routines for receivingand processing images for indexing and recognition. In one embodiment,the image processor 203 can be a set of instructions executable by theprocessor 235 to provide the functionality described below forprocessing the images for indexing and recognition. In anotherembodiment, the image processor 203 can be stored in the memory 237 ofthe computing device 200 and can be accessible and executable by theprocessor 235. In either embodiment, the image processor 203 can beadapted for cooperation and communication with the processor 235, thematching engine 205 and other components of the computing device 200 viasignal line 224.

In one embodiment, the image processor 203 receives images of contentfrom a user administrator. The content can be associated with anadvertising medium. The advertising medium can be any medium (print orvideo) through which an advertisement for a product is shown. Forexample, the images can be image frames from a car insuranceadvertisement on basic cable, a television drama episode on prime cable,a video clip on a video sharing website, a stream of an on-demandinternet streaming media provider, a video playback of a Blu-ray™ disc,etc. In another example, the images can be one of an image of a tabletPC advertisement page on a paper magazine, a front cover of a book, anarticle in a newspaper, etc.

In one embodiment, the image processor 203 requests the user to addinformation about the image as metadata for storing in the data storage243. For example, the name, type (daily newspaper, weekly magazine,basic cable, premium cable, etc.) and genre (business magazine, sportsmagazine, documentary channel, drama television channel, action movies,etc.) of the advertising medium from which the image is captured, theproduct identified in the image (e.g., car, clothing suit and sunglassesworn by a model, hand bag and earrings worn by an actress, etc.). Inanother embodiment, the image processor 203 identifies the metadata fromthe image itself. For example, the image processor 203 detects logo ofthe video channel in a received image frame, determines the season andepisode of the television drama, identifies the wardrobe worn by actorsin that episode, etc. and associates such information as metadata of thereceived image. In another example, the image processor 203 detects abrand name of a watch from the received image of a newspaperadvertisement, determines the name of the newspaper, page number of theadvertisement in the newspaper, etc. and associates such information asmetadata of the received image. In one embodiment, the image processor203 categorizes one or more products identified in the image asmetadata. For example, images including clothing, watches, sunglasses,handbag, etc. are categorized as belonging to the fashion category inthe metadata, images including cars, motorbikes, boats, etc. arecategorized as belonging to the vehicle category, and images includingsmartphones, tablet PCs, wearable gadgets, etc. are categorized asbelonging to the hardware category in the metadata, etc.

In one embodiment, the image processor 203 indexes the received imagesincluding the identified products, product categories and otheridentified metadata in the data storage 243. The indexing of thereceived images makes them searchable during image matching againstcaptured images by one or more users. In one embodiment, the imageprocessor 203 maps each image to a set of values, such as, product name,advertising medium, brand name, genre, etc. in Extensible MarkupLanguage (XML) file format.

In one embodiment, the image processor 203 sends data including theindexed set of images along with the associated metadata describing theset of images to the matching engine 205. In another embodiment, theimage processor 203 stores the data including the indexed set of imagesalong with the associated metadata describing the set of images in thedata storage 243. In one embodiment, each frame of a video is indexed.In another embodiment, unique frames of a video are indexed. Forexample, the data storage 243 indexes images when there is a thresholddifference between frames, such as a video of a person walking asopposed to a five second still shot of an object. Persons of ordinaryskill in the art will recognize that many different types of indexingtechniques are possible.

The matching engine 205 can be software including routines foridentifying a captured image and retrieving metadata describing thecaptured image. In one embodiment, the matching engine 205 can be a setof instructions executable by the processor 235 to provide thefunctionality described below for matching a captured image. In anotherembodiment, the matching engine 205 can be stored in the memory 237 ofthe computing device 200 and can be accessible and executable by theprocessor 235. In either embodiment, the matching engine 205 can beadapted for cooperation and communication with the processor 235, theimage processor 203, the user profile engine 207 and other components ofthe computing device 200 via signal line 226.

In one embodiment, the matching engine 205 receives a captured imagefrom a user including a time and location of capture. In one embodiment,the captured image can be received from the user device 115, forexample, a smartphone, a tablet PC, etc. and may include one or moreproducts. For example, the captured image could be an image frame from atelevision drama showing an actress with a handbag on the television. Inanother embodiment, the captured image could be captured directly fromthe media player 107, for example, if the user is viewing a video on asmartphone. In another example, the captured image could be an image ofan advertisement for a formal wrist watch in a daily newspaper. Thematching engine 205 identifies an indexed image in the data storage 243matching the captured image. In another embodiment, the matching engine205 retrieves metadata from the data storage 243 describing the capturedimage and associating the metadata with the captured image. For example,the matching engine 205 identifies the captured image of an actress in atelevision drama and associates it with retrieved metadata. The metadatadescribes that the captured image is from Season 3, Episode 2, broadcaston A2B channel, at 9 PM Eastern Standard Time, the identified product inthe captured image is a Louis Vuitton™ handbag, etc. In another example,the matching engine 205 identifies the captured image of anadvertisement for a formal wrist watch and associates it with retrievedmetadata. The metadata describes that the captured image is from page 5of a daily newspaper XYZ Herald dated Sunday 12, Jan. 2014, advertisingabout PQR wrist watch, and the image was captured at 09:15 AM, etc.

In one embodiment, the matching engine 205 sends data including thecaptured image matching the indexed image along with the associatedmetadata describing the captured image to the user profile engine 207for generating a user profile. The user profile engine 207 is explainedin more detail below. In another embodiment, the matching engine 205sends the data including the captured image matching the indexed imagealong with the associated metadata describing the captured image to theadvertisement engine 211 for delivering targeted advertisements inresponse. The advertisement engine 211 is explained in more detailbelow. In yet another embodiment, the matching engine 205 stores thedata including the identified image including products in the datastorage 243.

The user profile engine 207 can be software including routines forgenerating and updating user profiles for users capturing images. In oneembodiment, the user profile engine 207 can be a set of instructionsexecutable by the processor 235 to provide the functionality describedbelow for generating and updating the user profiles. In anotherembodiment, the user profile engine 207 can be stored in the memory 237of the computing device 200 and can be accessible and executable by theprocessor 235. In either embodiment, the user profile engine 207 can beadapted for cooperation and communication with the processor 235, theimage processor 203, the matching engine 205, the grouping engine 209and other components of the computing device 200 via signal line 228.

In one embodiment, the user profile engine 207 generates user profilesfor users based on their pattern in capturing images across theadvertising media. In one embodiment, the user profile engine 207receives an identification of a captured image associated with a userfrom the matching engine 205 and generates a user profile for the usercapturing the image. The user profile built for the user primarilyincludes user demographics, user interests and product buying habits. Inanother embodiment, the user profile engine 207 generates a user profilefor the user based on receiving user input at a time of initial userregistration. For example, the user profile engine 207 receives age,gender and location of the user at the initial user registration andadds such demograph information to the user profile. In one embodiment,the user profile engine 207 infers an interest of the user in aparticular subject or entity from identifying a pattern in which theuser captures images across the print and video media. The user profileengine 207 generates a user profile for the user based on the metadataof the captured images. The user profile engine 207 stores informationabout the images previously captured by the user including theassociated metadata in the user profile. For example, the user profileengine 207 identifies that a user captures images of actors withfashionable attire and accessories broadcast on a television channel.The user profile engine 207 stores information about the captured imagesincluding metadata in the user profile of that user inferring that theuser is interested in fashion. In another example, the user profileengine 207 identifies that a user captures images that are of articlesor advertisement pages from a technology magazines that include imagesof wearable fitness devices. The user profile engine 207 stores thecaptured images including metadata in the user profile of that userinferring that the user is interested in wearable fitness devices and,based on a trend, an early adopter of technology. In one embodiment, theuser profile engine 207 retrieves the stored user profile in response toreceiving a captured image from the user.

In one embodiment, the user profile engine 207 updates the user profilefor a user based on feedback information received from the advertisementengine 211. The advertisement engine 211 retrieves information about aproduct identified in a captured image by the user and delivers productadvertisements to the user in response to the captured image. Theadvertisement engine 211 is explained in greater detail below. In oneembodiment, the user profile engine 207 determines how the userinteracts with an advertised product from the feedback informationreceived from the advertisement engine 211 and updates the user profileaccordingly. For example, if the user clicks on advertised products, theuser profile engine 207 stores the click history associated with theadvertised products in the user profile. In another example, if the userbuys the advertised products, the user profile engine 207 stores thepurchase history associated with the advertised products and buyinghabits associated with the user in the user profile. In anotherembodiment, the user profile engine 207 monitors any subsequent activityof the user starting from capturing an image to clicking on anadvertised product associated with the captured image and updates theuser profile accordingly. For example, the user profile engine 207determines how much time the user spent buying an advertised product,how the user went about selecting a particular advertised product out ofothers to buy, etc. In another example, the user profile engine 207determines how much time elapsed between the user capturing an image andclicking on an advertised product associated with the captured image.The user profile engine 207 infers the user's interest in the advertisedproduct or brand based on whether the elapsed time is shorter or longer.

In one embodiment, the user profile engine 207 sends the user profilesto the grouping engine 209 for categorizing the user profiles into oneor more groups. In another embodiment, the user profile engine 207stores the data including the user profiles generated in the datastorage 243.

The grouping engine 209 can be software including routines for groupinguser profiles into one or more categories. In one embodiment, thegrouping engine 209 can be a set of instructions executable by theprocessor 235 to provide the functionality described below for groupingthe user profiles. In another embodiment, the grouping engine 209 can bestored in the memory 237 of the computing device 200 and can beaccessible and executable by the processor 235. In either embodiment,the grouping engine 209 can be adapted for cooperation and communicationwith the processor 235, the user profile engine 207, the advertisementengine 211 and other components of the computing device 200 via signalline 230.

In one embodiment, the grouping engine 209 receives a user profilegenerated for a user by the user profile engine 207 and categorizes theuser into one or more groups based on the user profile.

In one embodiment, the grouping engine 209 categorizes the user profilesbased on the users' interest in one or more products as identified fromtheir user profiles. For example, users engaged in capturing imagesrelating to fashion and accessories, such as, clothing, earrings, watch,handbag, gloves, sunglasses, etc. can be categorized as a groupinterested in fashion. In another example, users engaged in capturingimages relating to a particular genre of entertainment, such as, actionmovies, can be categorized as a group interested in action movie genreor that television channel showing action movies. In a third example,users engaged in capturing images relating to articles, advertisement,cover page, etc. in a business magazine can be categorized as a groupinterested in business.

In one embodiment, the grouping engine 209 categorizes the user profilesbased on the users' preferred advertising medium to use for capturingimages from as identified through their user profiles. For example,users predominantly engaged in capturing images from video such as,premium cable dramas, on-demand internet streaming websites, videosharing websites, etc. can be categorized as belonging to a group thatprefers video as a medium. In another example, users predominantlyengaged in capturing images from books, magazines, newspapers, etc. canbe categorized as belonging to a group that prefers print as a medium.In another embodiment, the grouping engine 209 categorizes the userprofiles based on the users' preference of an individual advertisingchannel in their already preferred medium to capture images. Forexample, user profiles that indicate the captured images as being fromproduct advertisements shown during a comedy sitcom can be categorizedas a group that watches that comedy sitcom. In another example, userprofiles that indicate the captured images as being from productadvertisement pages from a business magazine can be categorized as groupthat reads that business magazine. As a third example, user profiles canalso be further categorized as a group that watches comedy sitcoms onlyon A2B channel or a group that reads business articles only on Richiebusiness magazine.

In one embodiment, the grouping engine 209 categorizes the user profilesbased on the users' time of engagement with an advertising medium forcapturing images as identified from their user profiles. For example,users engaged in watching prime time television from 7 PM-10 PM andcapturing images of programs, product advertisements, etc. broadcastduring that time period can be categorized as a prime-time group. Inanother example, users engaged in watching news in the early morningbetween 6 AM-10 AM and capturing images of programs, product placements,etc. broadcast during that time period can be categorized as an earlymorning group.

In one embodiment, the grouping engine 209 categorizes the users basedon their product buying habits and patterns as identified through theuser profiles. For example, the grouping engine 209 identifies that someusers buy a product that is advertised in response to whenever receivinga captured image from the user. The grouping engine 209 categorizes suchusers as being part of an aggressive buyers group. In another example,the grouping engine 209 identifies that some users buy the product whenit is advertised only after a third or a fourth time. The groupingengine 209 categorizes such users as being part of a cautious buyersgroup or a thoughtful buyers group.

In one embodiment, the grouping engine 209 categorizes the users basedon determining the users' rate of adopting products and technology asidentified from their user profiles. In one embodiment, the groupingengine 209 categorizes the users into a group from one of the following:early adopters, early majority, late majority and laggards. For example,the grouping engine 209 identifies that some users express interest in aparticular product or product category by capturing images, receivinginformation about the product or category, and quickly making a purchasewhenever an advertisement for the product is delivered or on a launchdate for the product. The grouping engine 209 determines that such usersare predominantly younger who closely follow the particular product orproduct category and categorizes such users as an early adopters group.In a second example, the grouping engine 209 identifies that some usersexpress interest in the product by capturing images and buying theproduct only after a varying degree of time (significantly longer thanthe early adopters group). The grouping engine 209 categorizes suchusers as an early majority group. In a third example, the groupingengine 209 identifies that some users buy the product after the averagemember of the population has already adopted the product. The groupingengine 209 categorizes such users as a late majority group. In a fourthexample, the grouping engine 209 identifies that some users are alwaysthe last to adopt a particular product or category. The grouping engine209 determines that such users are predominantly advanced in age andaverse to change. The grouping engine 209 categorizes such users aslaggards.

In one embodiment, the grouping engine 209 sends the categorization ofusers into groups to the advertisement engine 211 for deliveringtargeted advertisements. In another embodiment, the grouping engine 209stores the categorization of users into groups in the data storage 243.

The advertisement engine 211 can be software including routines fordelivering targeted advertisements in response to users capturingimages. In one embodiment, the advertisement engine 211 can be a set ofinstructions executable by the processor 235 to provide thefunctionality described below for delivering targeted advertisements. Inanother embodiment, the advertisement engine 211 can be stored in thememory 237 of the computing device 200 and can be accessible andexecutable by the processor 235. In either embodiment, the advertisementengine 211 can be adapted for cooperation and communication with theprocessor 235, the matching engine 205, the user profile engine 207, thegrouping engine 209 and other components of the computing device 200 viasignal line 232.

In one embodiment, the advertisement engine 211 receives anidentification of the image captured by a user from the matching engine205. The identification includes retrieved metadata describing thecaptured image. In one embodiment, the advertisement engine 211 providesthe user with information about one or more products identified in thecaptured image based on the metadata describing the captured image. Theimage can include the product itself or other information about theproduct, such as text identifying a product. For example, if thecaptured image is of a smartphone advertisement off the video medium,the advertisement engine 211 retrieves information including thetechnological specification, launch date, price, retailer, etc.associated with the smartphone product and displays the retrievedinformation on the user device 115. In another embodiment, theadvertisement engine 211 instructs the user interface 211 to display theinformation about one or more products identified in the captured imageby generating hotspots on the captured image and linking the identifiedproducts to the retrieved information. For example, if the capturedimage is from a newspaper advertisement of a car insurance agent posingwith a car, the advertisement engine 211 creates hotspots on clear,visible spots on the car to provide information about the carinsurance's website, location address, telephone number, additionalvideos, etc.

In one embodiment, the advertisement engine 211 personalizes and selectsone or more advertisements from a set of advertisements to deliver tothe user capturing images based on the user profile of the user andmetadata describing the captured image. In one embodiment, theadvertisements relate to one or more identified products in the capturedimage. In another embodiment, the advertisements are opportunistic andcontextual to the theme of the captured image. For example, theadvertisement engine 211 identifies that a user regularly watches stockmarket news analysis in the evening from 6 PM-9 PM based on his userprofile. If the user is watching a movie based on his recent imagecapture received between 6 PM-9 PM on some occasion, the advertisementengine 211 delivers an advertisement, such as ticket advertisements,movie DVDs, etc. during that time period for movies with “Wall Street”themes to woo the user currently engaged in watching movies whootherwise is uninterested in movies. In one embodiment, theadvertisement engine 211 personalizes and selects the advertisements fordelivery based on the user demographics identified from the userprofile. For example, the advertisement engine 211 delivers oneadvertisement that is personalized for a 24 year old car owner andanother advertisement that is personalized for a 55 year old car ownerbased on each capturing an image of the car insurance advertisement pagein the newspaper. In another example, the advertisement engine 211delivers an advertisement for a handbag shopper based in Boston withcontact information of the local handbag branch in Boston and not SanFrancisco's branch.

In one embodiment, the advertisement engine 211 determines a user's timewatching a television show, time watching one television channel, timeof capturing images during the television show, genre interests, etc.based on the user profile. In one embodiment, the advertisement engine211 identifies an opportune time for delivering the advertisement to theuser to ensure optimal user engagement and effective advertising basedon the determined inferences from the user profile and the metadataretrieved for the captured image. For example, the advertisement engine211 delivers targeted advertisements to the user anytime during an hourlong television show. In another example, the advertisement engine 211delivers targeted advertisements to the user during advertisement breaksof the hour long show. In yet another example, the advertisement engine211 delivers targeted advertisements to the user at the end of the hourlong show. In another embodiment, the advertisement engine 211identifies a user's mood based on the user profile and the metadataretrieved for the captured image and delivers advertisements based onthe user's mood. For example, the advertisement engine 211 determinesthat the user is currently watching a romantic movie and deliversadvertisements for romantic movie recommendations, dating websites, etc.to the user.

In one embodiment, the advertisement engine 211 receives user profilescategorized into one or more groups from the grouping engine 209 anddelivers advertisements targeting the groups. In one example, theadvertisement engine 211 delivers an advertisement for a new room sprayproduct to users categorized in the “aggressive buyers” group for buyingan advertised product quickly. In another example, the advertisementengine 211 delivers an advertisement for a new platform-based smartphoneto users categorized in the “early adopters” group for closely followingsmartphone industry. In a third example, the advertisement engine 211delivers an advertisement for a new breakfast menu to users categorizedin the “early morning” group for engaging in watching product placementadvertisements in early morning video viewing hours.

In one embodiment, the advertisement engine 211 provides feedbackinformation to the user profile engine 207 responsive to the userclicking and/or buying the advertised products. The feedback informationincludes time spent buying an advertised product, the advertised productthat was bought, when the advertised product was bought, etc. In oneembodiment, the user profile engine 207 updates the user profiles basedon the feedback information received from the advertisement engine 211.

In one embodiment, the advertisement engine 211 sends the data includingthe advertisements to the user interface engine 213 for generating auser interface to deliver the advertisements. In another embodiment, theadvertisement engine 211 stores the data including the advertisements inthe data storage 243.

The user interface engine 213 is software including routines forgenerating graphical data for providing user interfaces to users. In oneembodiment, the user interface engine 213 is a set of instructionsexecutable by the processor 235 to provide the functionality below forgenerating graphical data for providing the user interfaces. In anotherembodiment, the user interface engine 213 is stored in the memory 237and is accessible and executable by the processor 235. In eitherembodiment, the user interface engine 213 is adapted for cooperation andcommunication with the processor 235 and other components of thecomputing device 200 via signal line 234.

In one embodiment, the user interface engine 213 receives instructionsfrom the advertisement engine 211 to generate a graphical interface thatdisplays information associated with one or more products identified onthe captured image to the user. In another embodiment, the userinterface engine 213 receives instructions from the advertisement engine211 to display advertisements for products related to the captured imagefor the user. The user interface engine 213 sends the graphical data toan application (e.g., a browser) in the user device 115 via thecommunication unit 241 causing the application to display the dataincluding the advertisements on a user interface.

Example Methods and Graphic Representations

FIG. 3 is a flow diagram 300 of one embodiment of a method for indexingimages. The media application 103 includes an image processor 203. Theimage processor 203 receives 302 images from a user and identifies 304 asource for each image. For example, the image processor 203 receives animage of an actress wearing a designer handbag and identifies that theimage is captured from a television program. The image processor 203identifies 306 one or more products in each image and categorizes 308the one or more products. For example, the image processor 203identifies that the handbag worn by the actress in the captured image isLouis Vuitton™ and categorizes the product as a fashion accessory. Theimage processor 203 indexes 310 each image including the one or moreproducts and product category. For example, the image processor 203 mapseach image to a set of values, such as, product name, advertisingmedium, brand name, genre, etc. in the data storage 243.

FIG. 4 is a flow diagram 400 of one embodiment of a method forcategorizing the users into one or more groups. The media application103 includes a matching engine 205, the user profile engine 207, thegrouping engine 209 and the advertisement engine 211. The matchingengine 205 receives 402 captured image from a user. For example, thematching engine 205 receives an image captured on a user's smartphone ofan actress wearing a pair of earrings on a television drama. In oneembodiment, the captured image includes a time and location of capture.In another embodiment, the captured images can be from any advertisingmedium, video or print. The matching engine 205 identifies 404 anindexed image matching the captured image and retrieves 406 metadatadescribing the captured image responsive to identifying the indexedimage. For example, the matching engine 205 identifies the capturedimage of the actress in the television drama and associates it withretrieved metadata. The metadata describes that the captured image isfrom Season 3, Episode 2, broadcast on A2B channel, at 9 PM EasternStandard Time, identified product in the captured image as Gucci™earrings, etc. The user profile engine 207 generates 408 a user profilefor the user based on the metadata describing the captured image and ahistory of at least one previously captured image associated with theuser. For example, the user profile engine 207 identifies that the usercaptures images of actors with fashionable attire and accessoriesbroadcast on television channels based on the metadata of capturedimages. The user profile engine 207 stores the captured images includingmetadata in the user profile generated for that user. The advertisementengine 211 identifies 410 a product in the captured image based on themetadata describing the captured image and provides 412 the user withinformation about the product. For example, the advertisement engine 211identifies the product Gucci™ earrings in the captured image based onthe metadata and retrieves information including the designer, price,store phone number, website address, etc. associated with the earringsand displays the retrieved information on the user's smartphone display.In one embodiment, the retrieved information is displayed as hotspots onthe captured image for the user. The user profile engine 207 updates 414the user profile based on how the user interacts with the product. Forexample, the user profile engine 207 determines if the user clicked onthe hotspots and stores the click history associated with the identifiedproducts in the user profile. In another example, if the user buys theadvertised products, the user profile engine 207 stores the purchasehistory associated with the user in the user profile. The groupingengine 209 categorizes 416 the user into one or more groups based on theuser profile. For example, since the user is engaged in capturing imagesrelating to fashion and accessories, such as, clothing, earrings, watch,handbag, gloves, sunglasses, etc. the grouping engine 209 categorizesthe user in a group interested in fashion.

FIG. 5 is a graphical representation 500 of one embodiment that displaysinformation associated with identified products on a captured image. Inone embodiment, the graphical representation 500 can be displayed on auser device 115, for example a smartphone associated with the usercapturing the image. In this graphic representation 500, the exampleimage is a captured image 501 of an actress wearing fashion accessories.The graphical representation 500 includes a hotspot 503 embedded on theidentified products in the captured image 501. The hotspot 503 alsoincludes a text description 505 for identifying the hotspot. Uponclicking the hotspot 503 the user is taken to the web page of thestylist. In the graphical representation 500, the captured image 501includes a clothing store hotspot 507 embedded on the blouse of theactress, a handbag store hotspot 509 embedded on the handbag of theactress, and a shoe store hotspot 511 embedded on the shoes of theactress.

FIG. 6 is a flow diagram 600 of one embodiment of a method fordelivering the user with an advertisement. The media application 103includes a matching engine 205, the user profile engine 207, and theadvertisement engine 211. The matching engine 205 receives 602 acaptured image including information about a product from a user. Forexample, the matching engine 205 receives an image captured on a user'ssmartphone of a movie shown on a television channel that includes theproduct itself, information for purchasing the product, etc. In anotherexample, the matching engine 205 receives an image captured on theuser's smartphone of an article in a business magazine. The matchingengine 205 identifies 604 an indexed image matching the captured imageincluding information about the product and retrieves 606 metadatadescribing the captured image including information about the productresponsive to identifying the indexed image. For example, the matchingengine 205 identifies an indexed image matching the captured image fromthe data storage 243. The metadata describing the captured image couldinclude that the captured image is from XYZ movie broadcast on “A2Bchannel”, etc. In another example, the metadata describing the capturedimage could include that the captured image is from page 6 of Richiebusiness magazine issued in the month of December and the article istitled “Falling gold prices,” etc. The user profile engine 207 retrieves608 a user profile for the user. In one embodiment, the user profile forthe user primarily includes user demographics, user interests andproduct buying habits. The user profile engine 207 stores all the imagespreviously captured by the user including the associated metadata in theuser profile. The advertisement engine 211 identifies 610 anadvertisement from a set of advertisements based on the user profile andthe metadata describing the captured image including information aboutthe product and provides 612 the user with the advertisement based onthe user profile. In one embodiment, the advertisement can beopportunistic and contextual to the theme of the captured image. Forexample, the advertisement engine 211 identifies that the user capturedan image of an article titled “Falling gold prices” from the metadataretrieved for the captured image. The advertisement engine 211 selectsan advertisement for a gold chain necklace from a designer store basedon the context identified for the captured image and displays theadvertisement on the user's smartphone.

FIG. 7 is a graphic representation 700 of one embodiment for deliveringan advertisement to the user capturing an image. In the illustratedembodiment, the graphical representation 700 includes a television 701(i.e., media player 107) and a tablet PC 703 (i.e. user device 115). Theuser watching a super hero movie on the television 701 captures an imageof a climactic scene in the movie 705 using the tablet PC 703. The useris then delivered an advertisement 707 on the tablet 703 on which theuser captured the image. The advertisement 707 displays that there aretwo movies that can be purchased for just $5 with the offer expiring onJan. 12, 2014.

The foregoing description of the embodiments has been presented for thepurposes of illustration and description. It is not intended to beexhaustive or to limit the specification to the precise form disclosed.Many modifications and variations are possible in light of the aboveteaching. It is intended that the scope of the embodiments be limitednot by this detailed description, but rather by the claims of thisapplication. As will be understood by those familiar with the art, theexamples may be embodied in other specific forms without departing fromthe spirit or essential characteristics thereof. Likewise, theparticular naming and division of the modules, routines, features,attributes, methodologies and other aspects are not mandatory orsignificant, and the mechanisms that implement the description or itsfeatures may have different names, divisions and/or formats.Furthermore, as will be apparent to one of ordinary skill in therelevant art, the modules, routines, features, attributes, methodologiesand other aspects of the specification can be implemented as software,hardware, firmware or any combination of the three. Also, wherever acomponent, an example of which is a module, of the specification isimplemented as software, the component can be implemented as astandalone program, as part of a larger program, as a plurality ofseparate programs, as a statically or dynamically linked library, as akernel loadable module, as a device driver, and/or in every and anyother way known now or in the future to those of ordinary skill in theart of computer programming. Additionally, the specification is in noway limited to embodiment in any specific programming language, or forany specific operating system or environment. Accordingly, thedisclosure is intended to be illustrative, but not limiting, of thescope of the specification, which is set forth in the following claims.

What is claimed is:
 1. A computer-implemented method comprising:receiving, with one or more processors, a captured image from a user;identifying, with the one or more processors, an indexed image matchingthe captured image; retrieving, with the one or more processors,metadata describing the captured image responsive to identifying theindexed image; generating, with the one or more processors, a userprofile for the user based on the metadata describing the captured imageand a history of at least one previously captured image associated withthe user; identifying, with the one or more processors, a product in thecaptured image based on the metadata describing the captured image;providing, with the one or more processors, the user with informationabout the product; updating, with the one or more processors, the userprofile based on how the user interacts with the product; andcategorizing, with the one or more processors, the user into one or moregroups based on the user profile.
 2. The method of claim 1, wherein themetadata describing the captured image includes a time and locationassociated with the captured image, a genre of the captured image, atype of advertising medium associated with the captured image, a name ofadvertising medium associated with the captured image, and a name of theproduct in the captured image.
 3. The method of claim 1, wherein theuser profile for the user includes age, gender, location, timestamp ofthe at least one previously captured image, metadata of the at least onepreviously captured image, history of products previously interactedwith by the user, history of products previously purchased by the userand time spent by the user shopping for products.
 4. The method of claim1, wherein the user profile for the user is generated at a time ofinitial user registration.
 5. The method of claim 1, wherein providingthe user with information about the product further comprises displayinghotspots on the captured image linking the identified product to theproduct's website.
 6. The method of claim 1, wherein categorizing theuser into one or more groups based on the user profile further comprisescategorizing by the user's time of engagement with an advertisingmedium.
 7. The method of claim 1, wherein categorizing the user into oneor more groups based on the user profile further comprises categorizingby the user's rate of product adoption.
 8. The method of claim 1,wherein categorizing the user into one or more groups based on the userprofile further comprises categorizing by the user's shopping patternand the user's interest in products.
 9. The method of claim 1, whereincategorizing the user into one or more groups based on the user profilefurther comprises categorizing by the user's preference of a medium. 10.A system comprising: one or more processors; a matching engine stored ona memory and executable by the one or more processors, the matchingengine for receiving a captured image from a user, identifying anindexed image matching the captured image and retrieving metadatadescribing the captured image responsive to identifying the indexedimage; an advertisement engine stored on the memory and executable bythe one or more processors, the advertisement engine coupled to thematching engine for identifying a product in the captured image based onthe metadata describing the captured image and providing the user withinformation about the product; a user profile engine coupled stored onthe memory and executable by the one or more processors, the userprofile engine coupled to the matching engine and the advertisementengine for generating a user profile for the user based on the metadatadescribing the captured image and a history of at least one previouslycaptured image associated with the user; and a grouping engine coupledto the user profile engine for categorizing the user into one or moregroups based on the user profile.
 11. The system of claim 10, whereinthe user profile for the user includes age, gender, location, timestampof the at least one previously captured image, metadata of the at leastone previously captured image, history of products previously interactedwith by the user, history of products previously purchased by the userand time spent by the user shopping for products.
 12. The system ofclaim 10, wherein the advertisement engine providing the user withinformation about the product further comprises displaying hotspots onthe captured image linking the identified product to the product'swebsite.
 13. The system of claim 10, wherein the grouping enginecategorizing the user into one or more groups based on the user profilefurther comprises categorizing by the user's time of engagement with anadvertising medium.
 14. The system of claim 10, wherein the groupingengine categorizing the user into one or more groups based on the userprofile further comprises categorizing by the user's rate of productadoption.
 15. The system of claim 12, wherein the grouping enginecategorizing the user into one or more groups based on the user profilefurther comprises categorizing by the user's preference of a medium. 16.A computer program product comprising a computer useable mediumincluding a computer readable program, wherein the computer readableprogram when executed on a computer causes the computer to: receive acaptured image from a user; identify an indexed image matching thecaptured image; retrieve metadata describing the captured imageresponsive to identifying the indexed image; generate a user profile forthe user based on the metadata describing the captured image and ahistory of at least one previously captured image associated with theuser; identify a product in the captured image based on the metadatadescribing the captured image; provide the user with information aboutthe product; update the user profile based on how the user interactswith the product; and categorize the user into one or more groups basedon the user profile.
 17. The computer program product of claim 16,wherein the metadata describing the captured image includes a time andlocation associated with the captured image, a genre of the capturedimage, a type of advertising medium associated with the captured image,a name of advertising medium associated with the captured image, and aname of the product in the captured image.
 18. The computer programproduct of claim 16, wherein the computer readable program when executedon the computer causes the computer to provide the user with informationabout the product by displaying hotspots on the captured image linkingthe identified product to the product's website.
 19. The computerprogram product of claim 15, wherein the computer readable program whenexecuted on the computer causes the computer to categorize the user intoone or more groups based on the user profile by categorizing based onthe user's time of engagement with an advertising medium.
 20. Thecomputer program product of claim 15, wherein the computer readableprogram when executed on the computer causes the computer to categorizethe user into one or more groups based on the user profile bycategorizing based on the user's rate of product adoption.